Skip to content

Fix: Test memory leak and Failing test when md5 and sha are enabled - #11025

Open
aidankeefe2022 wants to merge 3 commits into
wolfSSL:masterfrom
aidankeefe2022:test-fix-for-md5-and-sha-enabled
Open

Fix: Test memory leak and Failing test when md5 and sha are enabled#11025
aidankeefe2022 wants to merge 3 commits into
wolfSSL:masterfrom
aidankeefe2022:test-fix-for-md5-and-sha-enabled

Conversation

@aidankeefe2022

Copy link
Copy Markdown
Member

Description

Fixed a memory leak where if _ret is already failing, the rngInit function will not run, causing rngFree to free garbage data. The fix was to have rngInit always run to match with rngFree by swapping ExpectInt to DoExpectInt. Fixed failing test where WC_SIG_MIN_HASH_TYPE was not properly set for --enable-sha and --enable-md5, so we skip under those conditions.

Found while testing wolfCLU

Testing

Code no longer leaks, and code no longer fails with the wrong code for the test when SHA and MD5 are enabled

@Frauschi Frauschi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🐺 Skoll Code Review

Overall recommendation: APPROVE
Findings: 6 total — 2 posted, 4 skipped

Posted findings

  • [Medium] Weak-hash guard is skipped even when WC_SIG_MIN_HASH_TYPE raises the floor, and silently self-destructs if the default ever moves into a headertests/api/test_signature.c:338
  • [Medium] Lowered-floor branch of wc_SignatureCheckHashStrength() has no coverage at alltests/api/test_signature.c:338
Skipped findings
  • [Low] PR description attributes the failure to --enable-sha/--enable-md5, but the trigger is --enable-wolfclu
  • [Low] Two different init/free-pairing conventions now coexist in this file
  • [Low] Weak-hash assertions skipped for any WC_SIG_MIN_HASH_TYPE override, including stronger floors
  • [Info] Guard depends on WC_SIG_MIN_HASH_TYPE's default living in signature.c, not a public header

Review generated by Skoll via Claude/Codex

Comment thread tests/api/test_signature.c Outdated
Comment thread tests/api/test_signature.c Outdated
@aidankeefe2022

Copy link
Copy Markdown
Member Author

Retest this please

@Frauschi Frauschi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🐺 Skoll Code Review

Overall recommendation: APPROVE
Findings: 2 total — 2 posted, 0 skipped

Posted findings

  • [Medium] Lowered-floor else branch at the generate site calls wc_SignatureVerify() instead of wc_SignatureGenerate()tests/api/test_signature.c:538-544
  • [Low] Guard compares wc_HashType enum values; the library compares digest sizes, and the two disagree in real configstests/api/test_signature.c:345,532

Review generated by Skoll via Claude/Codex

Comment thread tests/api/test_signature.c Outdated
Comment thread tests/api/test_signature.c Outdated
@aidankeefe2022
aidankeefe2022 force-pushed the test-fix-for-md5-and-sha-enabled branch from 6ae7980 to 41c6766 Compare August 4, 2026 21:59
@aidankeefe2022

Copy link
Copy Markdown
Member Author

Fix for failing tests was resolved by #11026 but fix for memory leak on failure is real and DoExpect for rng init should still go in

@Frauschi

Frauschi commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Jenkins retest this please

@Frauschi Frauschi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🐺 Skoll Code Review

Overall recommendation: APPROVE
Findings: 3 total — 1 posted, 2 skipped

Posted findings

  • [Medium] RNG fix deviates from the file's own (and the repo's dominant) zero-then-init idiomtests/api/test_signature.c:307
Skipped findings
  • [Low] PR description no longer matches the diff: the WC_SIG_MIN_HASH_TYPE half already landed on master
  • [Info] Fix is unverified by execution in this review environment

Review generated by Skoll via Claude/Codex


XMEMSET(data, 0x5A, sizeof(data));
ExpectIntEQ(wc_InitRng(&rng), 0);
DoExpectIntEQ(wc_InitRng(&rng), 0);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 [Medium] RNG fix deviates from the file's own (and the repo's dominant) zero-then-init idiom
💡 SUGGEST convention

The fix is correct: wc_FreeRng() dereferences rng->status, rng->drbg, rng->drbg512 and hands rng->heap to XFREE() (wolfcrypt/src/random.c:2615-2680), so pairing an unconditional DoExpectIntEQ(wc_FreeRng(...)) with a conditional ExpectIntEQ(wc_InitRng(...)) really did free garbage off an uninitialized stack WC_RNG. And a failing wc_InitRng is safe to free, because _InitRng() does XMEMSET(rng, 0, sizeof(*rng)) at random.c:1838 before anything can fail. So there is no bug here.

The concern is that the PR fixes the asymmetry from the wrong end, and in a way this file uses nowhere else. Every other handle in test_signature.c follows the standard idiom -- zero the struct, init conditionally, free unconditionally: ecc_key (XMEMSET(&ecc, 0, ...) at lines 59/253/686 + ExpectIntEQ(wc_ecc_init(...)) + DoExpectIntEQ(wc_ecc_free(...))), RsaKey (lines 104/555/742), and falcon_key (line 182). The zeroing is precisely what makes those unconditional frees safe. WC_RNG is now the sole handle in the file that is left uninitialized and instead relies on the init macro never being downgraded.

That also matches the wider repo: across tests/api/, XMEMSET(&rng, 0, sizeof(rng)) appears 226 times in 29 files and ExpectIntEQ(wc_InitRng 324 times, while DoExpectIntEQ(wc_InitRng appears only in these 7 new lines. A zeroed WC_RNG is safe to free -- WC_DRBG_NOT_INIT is 0 and WC_DRBG_BANKREF is 4 (wolfssl/wolfcrypt/random.h:346-352), so the WC_RNG_BANK_SUPPORT early-return in wc_FreeRng is not tripped.

Secondary effect of the chosen approach: wc_InitRng() now instantiates a DRBG and draws entropy even when the test has already failed and every following assertion will be skipped. Harmless on CI, but it is wasted work on entropy-constrained targets and is the opposite of what the skip machinery is for.

Suggestion:

Suggested change
DoExpectIntEQ(wc_InitRng(&rng), 0);
XMEMSET(&rng, 0, sizeof(rng));
DoExpectIntEQ(wc_InitRng(&rng), 0);

Recommendation: Prefer adding XMEMSET(&rng, 0, sizeof(rng)); at each of the 7 sites (lines 185, 307, 404, 481, 642, 691, 777). Either keep ExpectIntEQ for the init -- matching the 324 other call sites and the ecc/rsa/falcon handling in this very file -- or keep DoExpectIntEQ and add the memset anyway as defense in depth, so the free stays safe regardless of which macro a future refactor or merge-conflict resolution leaves behind. If the DoExpectIntEQ form is deliberately preferred going forward, say so in the PR so the repo can converge rather than carry two idioms. Note separately that the same conditional-init/unconditional-free asymmetry exists in other tests/api/ files that lack the memset -- worth a follow-up sweep, but out of scope here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants